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ABSTRACT ~ « ^ 

Evaluation activities should always be considered in 
terms of the effect: that they have on the persons or programs being 
evaluated. This is a particularly critical concept when young 
children are being evaluated. Evaluation should be of some value to 
the persons or programs being evaluated. For example, when a Follow 
Through program is being evaluated, persons in the community should 
be assisted in developing the skills they need to conduct evaluative 
activities valuable to them. In other words, every evaluation should 
be considered in terms of its effects on the individuals or programs 
being evaluated. Procedures commonly used to measure outcomes in 
early childhood programs are conceived too narrowly and place 
artificial constraints on naturalness and spontaneity. (Author/KM) 
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\ When evaluation time comes to a federally funded early childhood 
, program, everyone in the school knows It. Persons Involved are either 
required to change their roles or do so Inadvertantly because of the 
situation. The project director becomes a coordinator of testers, 



observers, tests and testing places. The social worker becomes a 
scheduler of parents' time, a transportation expert, and maybe even a 
taxi driver. Teachers are transformed Into authoritarian test givers 
or the self-conscious subject of a strange observer. Pupils may also 
be watched by a note taking stranger, or required to sit still and check 
a correct answer on a certain page In a certain amount of time. 

Rooms In the school change their functions. An advisor In one of 
our larger programs reported that use of the library was denied to the 
entire school for almost two months while the testing was In process. 
Books are not the only thing denied children In deference to evaluator's 
schedules. Field trips, often a culminating feature of the school year, 
must often be curtailed or eliminated due to the evaluation data being 
collected at that time. 

Since Follow Through Is a research and demonstration program, such 
changes In roles and schedules. If they do not produce spurious data to 
observer or test glver^ can be seen as essential Interruptions. 

The objection here Is not the Interruption per se, but who will 
benefit from It. This will be dealt with later In detail. 

There are other familiar objections to evaluation. Proponents of open 
education object to current evaluation techniques on several grounds. 
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Roland Barth suggests: 

The preferred source of verification for a child ^s solution 
to a problem comes through the materials he is working with, rather 
than through standardized tests. 

Perhaps the most frequent objection of all, is to these "objective tests," 
which are seen as being at such great variance with open education that they 
should not be used as measuring instruments. 

It appears obvious that when limited measuring instruments are applied to 
programs with broad and flexible goals the measurements provide only limited 
information about that educational model. Standard achievement tests, or 
even observational schedules which look only at a child's achievement in 
academic subjects won't tell anyone the real capacity the child may have for 
growing, thinking and behaving. 

" In evaluating the national evaluation of Pollow Through, other limitations 
-rarely considered are even more Important: 

Who evaluates? 

To whom is the evaluation addressed? 

What is the basis for the timing of the evaluation? 

What is the distance in thought, in status and in economic rewards 
between the evaluator 'and the evaluatee?-^ 

In order to understand the effect these limitations have on any evaluation 

of the program, the characteristics of open education need defining. Barth and 

Rathbone find "openness" in all aspects of the model: 

Not only are classroom doors ajar and children moving about, but 
time is open and so is curriculum. Most fundamental, however, is an 
"openness of self." Persons are openly sensitive to and supportive 
of other persons, not closed off by anxiety, threat, custom and role. 
Feelings are exposed, acknowledged and respected, not withheld in 
fear and defensiveness. Administrators are open to initiatives on 
the part of teachers; teachers are open to the possibilities inherent 
in children; children are open to the possibilities in other children, 
in materials, in themselves. 
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Two groups of persons Bartn and Rathbone did not Include are parents 
and professional evaluators. 

In many evaluations of educational programs, whatever the model, the 
Interests of parents are Ignored • Except for a few programs which Include 
them deliberately as crucial persons In Influencing a chilli's cognitive 
development, parents are rarely consulted by teachers, school administrators 
or evaluators about what they think ought to be in an educational program 
for their children. Seldom do they have input into when, why or how such 
a program might be evaluateo, albeit they usually know their children better 
than anyone. 

Clearly, openness in the classroom and school cannot be maximally effective 
for children if there is not also openness between home and school* Because 
parents, school personnel and community leaders are often not given an opportunity 
to propose, let alone respond, to either the purposes or the findings of national 
evaluators, such evaluations tend to reinforce suspicion and antagonism between 
the very groups — parents and teachers — who need to cooperate with one another 
for better education. 

Consider who usually requests a program evaluation in this country. In 
federally funded programs the request for an evaluation is instituted by Congress 
or a Government Department or Bureau, who are properly concerned that the tax 
dollars given to specific programs are will spent. Tax paying parents would, 
no doubt, support this purpose. However, national programs, such as Follow 
Through are now so geographically and educationally complex that only large, 
wealthy research and development firms are able to bid for the opportunity to 
evaluate them. Only such firms can afford the professional expertise, the 
computer time, and the travel money to provide a sophisticated research design 
which will produce a valid evaluation. Unfortunately, it is a long way from 
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an Appalachian cabin or a city slum tenement to the plush offices of a 

research Institute. Consider how such firms operate: In order to meet 

federal rules and deadlines they hire three shifts of secretaries and pay 

professionals thousands of dollars for their grantsmenshlp • One firm has an 

interesting Innovation-- namely a telephone in every toilet cubicle so no 

Important communication will be missed at any time. This firm has often had 

government contracts to evaluate programs sexrving families who had neither 

telephone nor Indoor toilet* 

Some distance between evaluators and school children might result in 

objective reports — but where the gap has reached such great proportions it 

is probable they cannot understand each other. Let me cite some examples: 

One of our Follow Through programs is in a tobacco growing country. 

Testers were sent in to interview or test parents in the fall and in the 

spring* A disappointed community member who cared a lot about the success o€ 

-the Follow Through program commented: 

We didn't have a very good showing of parents. When the evalu- 
ation team came in the fall, most parents were harvesting tobacco, 
when they returned in the spring, most parents were planting tobacco. 
It was reported we had poor parent involvement. 

Another time a team of experts came to evaluate a program in a racially 
mixed community. One of the experts reported poor relationships between whites 
and blacks because the white social worker did not go to the door of the black 
parent when picking her up for a neeting, but sat in the car and honked her 
horn* Had the "expert" openly confronted either party (instead of sending off 
a written evaluation to the government funding agency) she would have discovered 
both social worker and parent both felt so sure of their good relationship they 
did not stand on ceremony with one another* 

In another Follow Through program parents refused to cooperate with the 
evaluation team — not because it was made up of distant experts, but because the 
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distant experts had made the faux pas of hiring parents from a rival program. 
The Follow Through parents were convinced these parent testers had a vested 
interest in seeing that the Follow Through children did poorly on the tests 
and could manipulate the results. They reported this in confidence to an 
^visor feeling they had no way to communicate with the distant evaluators 
except through the distrusted testers. 

Perhaps for innovative, complex educational programs it is the timing of 
program Implementation and evaluation which is most unrealistic in terms of 
an Open Education model. Congress and most school systems are impossibly 
Ijnpatient. They demand to know at least each year, and in innumerable mid year 
reports what outcomes they are getting for money spent. This might be realistic 
if they were funding a circus. For example: In a few month's time it would be 
easy to list cost of training for two trapeze artists and three stunt dogs and 
describe exactly which tricks they had learned to perform; It would even be 
possible where you were dealing with school children if the concern was only 
with reading skills and computing ability. For example, 30 children had this 
kind of reading and arithmetic training and now they can read so many words per 
minute and multiply by three numbers. 

However, if you consider education as something beyond a few skills, and 
children as complex, lively human beings who need and want to know much more 
than how to read fast and compute accurately, then the problem of who should be 
evaluated and at what time , is important. 

In Great Britain one is told that it takes five years after college for a 
teacher to become a good implementer of open education. Now even if we do 
things faster in the U.S.A., it is still unlikely that in one year's time we 
will be able to train a traditional teacher to become an open education teacher. 
If she can't learn to teach the model in a year, why are we measuring outcomes 
before there is a meaningful input? 
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<5i^ The notion of yearly evaluations of pupil outcomes creates unnecessary 

tension and nuisance. Most teachers are aware when evaluations can give them 
valuable feedback. They also know this may vary in terms of the^r own skills, 
the program they are attempting to implement and the background of the children 
'Involved. Too often in open education programs the standard achievement test^ 
demanded at a certain time^ guarantees feelings of frustration or failure for 
children who most need to experience success. A good teacher wants feedback 
in the form of test scores or observational data. Such a teacher is eager 
to know how well she and her pupils are doing. However, that "teacher needs 
■the option of choosing when, and how to get this information. Her needs are 
rarely considered by a federal funding system which dictates requests for 
evaluation designs in ridiculously limited periods leaving no time for ^valuator 
or evaluated to communicate, reflect and respond in a reasonable way. 
To summarize the limitations: 

Who evaluates? Highly paid professionals in a limited number of 
Research and Development firms. 

To whom is the evaluation addressed? To Government agencies and 
in turn to Congress. 

What is the basis for the timing of the evaluation? Congressional 
funding habits. 

What is the distance in thought, in status and in economic rewards 
between the evaluated and the evaluatee? Immense. 

Is there a way to use federal funds to help rather than hinder 
evaluation? Yes. 

Let's envision a different evaluation procedure using the Open Education 
Advisory model. It might also require federal funds, but requests for their use 
would come from the tax payers who provided the funds in the first place. This 
model offers support and advice to people at their request and timed according 
to the needs of the program. 

Supposing the evaluation was initiated not by Congress, but by the teacher, 
gj^^ herself, who wanted to know what a child, or a class, knew in certain areas at 
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certain times. Further suppose she could request an evaluation specialist 
(funded by the government) to help her do this? 

Supposing parents, who were concerned that their children needed to learn 
faster or better, wanted to diagnose the problems their children had in 
learning^ or to discover appropriate techniques for alleviating these problems. 
Could our taxes go toward sending them a team of experts in diagnosing 
learning problems? ^ 

Supposing a school administrator wanted to look at different teaching 
styles and practices in his school system to determine which seemed to be the 
most effective for his school population. What about an expert evaluator to 
help him. do this? — someone who understands school administration and has ability 
to cooperate with teachers. 

Supposing a child wants to know something, e.g., how to measure the weight 
of an object using a scale, but has not yet had the opportunity and guidance to 
find the solution. If given a chance and some help, he will, no doubt, discover 
the solution. No need here for a national testing program to evaluate the 
child's failure or success. 

At this time in America such simple use of expert help for school personnel 
is rare. Time, space and guidance for the child who wants to look for solutions 
to problems is often lacking. 

This paper poses the problem: Can the federally funded $20,000 evaluator 
from the big city on the east or west coast ever come to know, understand, explain, 
communicate, and help the $3,000 family in the hills, or the $8,000 teacher in 
the inner city? 

Unless sophisticated evaluators protest the harmful constraints concerning 
the timing and procedure of their evaluations; unless they learn to listen to 
and address the persons they are evaluating, funds for national evaluations of 
federal programs must be viewed as still another chance for the rich to get 
richer and the poor to feel more defeated. 



